Goto

Collaborating Authors

 detect sarcasm


Assessing how hyperparameters impact Large Language Models' sarcasm detection performance

Gole, Montgomery, Miranskyy, Andriy

arXiv.org Artificial Intelligence

Sarcasm detection is challenging for both humans and machines. This work explores how model characteristics impact sarcasm detection in OpenAI's GPT, and Meta's Llama-2 models, given their strong natural language understanding, and popularity. We evaluate fine-tuned and zero-shot models across various sizes, releases, and hyperparameters. Experiments were conducted on the political and balanced (pol-bal) portion of the popular Self-Annotated Reddit Corpus (SARC2.0) sarcasm dataset. Fine-tuned performance improves monotonically with model size within a model family, while hyperparameter tuning also impacts performance. In the fine-tuning scenario, full precision Llama-2-13b achieves state-of-the-art accuracy and $F_1$-score, both measured at 0.83, comparable to average human performance. In the zero-shot setting, one GPT-4 model achieves competitive performance to prior attempts, yielding an accuracy of 0.70 and an $F_1$-score of 0.75. Furthermore, a model's performance may increase or decline with each release, highlighting the need to reassess performance after each release.


Was that Sarcasm?: A Literature Survey on Sarcasm Detection

Bagga, Harleen Kaur, Bernard, Jasmine, Shaheen, Sahil, Arora, Sarthak

arXiv.org Artificial Intelligence

Sarcasm is hard to interpret as human beings. Being able to interpret sarcasm is often termed as a sign of intelligence, given the complex nature of sarcasm. Hence, this is a field of Natural Language Processing which is still complex for computers to decipher. This Literature Survey delves into different aspects of sarcasm detection, to create an understanding of the underlying problems faced during detection, approaches used to solve this problem, and different forms of available datasets for sarcasm detection.


On Sarcasm Detection with OpenAI GPT-based Models

Gole, Montgomery, Nwadiugwu, Williams-Paul, Miranskyy, Andriy

arXiv.org Artificial Intelligence

Sarcasm is a form of irony that requires readers or listeners to interpret its intended meaning by considering context and social cues. Machine learning classification models have long had difficulty detecting sarcasm due to its social complexity and contradictory nature. This paper explores the applications of the Generative Pretrained Transformer (GPT) models, including GPT-3, InstructGPT, GPT-3.5, and GPT-4, in detecting sarcasm in natural language. It tests fine-tuned and zero-shot models of different sizes and releases. The GPT models were tested on the political and balanced (pol-bal) portion of the popular Self-Annotated Reddit Corpus (SARC 2.0) sarcasm dataset. In the fine-tuning case, the largest fine-tuned GPT-3 model achieves accuracy and $F_1$-score of 0.81, outperforming prior models. In the zero-shot case, one of GPT-4 models yields an accuracy of 0.70 and $F_1$-score of 0.75. Other models score lower. Additionally, a model's performance may improve or deteriorate with each release, highlighting the need to reassess performance after each release.


How to Use MLOps to Detect Sarcasm

#artificialintelligence

Sarcasm can be difficult to detect in text, especially for machines. However, with the power of large language models, it's possible to create a tool that can identify sarcastic comments with high accuracy. That's exactly what the ClearML team did with their latest project: a sarcasm detector that combines various ClearML tools to showcase the capabilities of MLOps. In the age of chatGPT and proprietary APIs, this project is meant as an example of how to create tools based on large language models that can run on your own machine, so you have full control over it. And thanks to ClearML being open source, even the whole MLOps stack can run locally.


Is that text SARCASTIC … ?🤔

#artificialintelligence

As we all know, Artificial Intelligence and Machine Learning are transforming the world. It has numerous applications in various fields, from medical science to video games. Areas such as E-Commerce and Social media have used AI lucratively and have benefited the most. However, in this project, I decided to use AI for a fun task. I tried to build a model to detect sarcasm in a text.


It's Easy for Computers to Detect Sarcasm, Right?

#artificialintelligence

So all the all the previous achievements by different machine learning algorithms, precisely the reinforcement learning algorithms, is that it could look at …


How did the University of Central Florida Develop a Sarcasm Detector?

#artificialintelligence

No second thoughts about the fact that how critical has social media become a part of our lives. We rely on social media so much that today imagining a life without it does not sink in. No wonder social media is considered to be one of the best platforms to market and sell different products and services in addition to being a dominant form of communication. With this platform, you stand a chance to reach out to the maximum lot. While this medium is used for driving sales, another area that it caters to is how are our customers reacting to what you are delivering.


Artificial Intelligence Can Now Detect Sarcasm. But For What?

#artificialintelligence

Artificial Intelligence is one step closer to being more human-like as it can now detect sarcasm. Funded by the U.S military, a new AI tool has managed to do a task that is tough for computer algorithms to perform in general, identifying the tone and irony of the human voice. This advancement can help intelligence agencies perform better trend analysis by identifying social media posts that are basically sarcastic in nature and meaning no harm. How did the AI tool figure it out? According to two researchers from the University of Central Florida, some words in a set of combinations can be a clear indicator of sarcasm in social media posts.


Salesforce releases AI tool to detect sarcasm in a tweet

#artificialintelligence

Local differences: If steps are not taken to lessen the rate of warming from climate change, counties in the South and lower Midwest -- which on average tend to already be poorer and warmer -- may lose as much as 20% of their income and may experience higher mortality rates. However, areas of the Pacific Northwest, the Great Lakes region and New England -- which on average tend to be wealthier and cooler -- could benefit economically from the change and see lower mortality rates. A climate impact map by county is available here. Dire warning: The researchers predict mortality will increase by 5.4 deaths per 100,000 people for every one degree Celsius rise in temperature. "We show there are going to be as many additional deaths from climate change as there are car crashes, and possibly more. Of the sectors we looked at, the greatest costs by far to society are going to come from those additional deaths," Rising told Axios.


Sarcasm Detection with Machine Learning in Spark

#artificialintelligence

This post is inspired by a site I found whilst searching for a way to detect sarcasm within sentences. As humans we sometimes struggle detecting sarcasm when we have a lot more contextual information available to us. People are emotive when they speak, they use certain tones and these traits can help us understand when someone is being sarcastic. However we don't always catch it! So how the hell could a computer detect this, when all it has is text.